System and method for a golf event using artificial reality

ABSTRACT

A system and method for creating content such as artificial reality (AR) messages at an event, particularly among members on a social network, thereby enhancing and expanding the event experience. Typically, a participant shares an event with spectators, such as friends or a subset of friends in the participant&#39;s social network. The AR message may include geo-referenced artificial reality words, products or symbols and appear in a perspective view of the event to the participant or spectators. In addition to creating an active gallery for an event, messages, audio and video can be exchanged among participants and spectators, and virtual goods, money, bets, applause, other feedback, and donations exchanged.

CONTINUING DATA

This application is a continuation of U.S. patent application Ser. No.14/037,642, filed Sep. 26, 2013, which is a continuation of U.S. patentapplication Ser. No. 13/605,174 (now U.S. Pat. No. 8,589,488) filed Sep.6, 2012 which is a continuation of U.S. patent application Ser. No.13/182,930 (now U.S. Pat. No. 8,933,967) filed Jul. 14, 2011 and Ser.No. 13/152,476 filed Jun. 3, 2011. Application '930 is acontinuation-in-part of U.S. patent application Ser. No. 12/111,688filed Apr. 29, 2008 which is a continuation-in-part of U.S. patentapplication Ser. No. 11/875,414 (now U.S. Pat. No. 8,249,626) filed Oct.19, 2007 which is a continuation-in-part of U.S. patent application Ser.No. 11/624,998 (now U.S. Pat. No. 8,275,397) filed Jan. 19, 2007 whichis a continuation-in-part of U.S. patent application Ser. No. 11/456,715(now U.S. Pat. No. 7,855,638) and Ser. No. 11/456,723 (now U.S. Pat. No.7,518,501) filed Jul. 11, 2006 both of which claim priority to U.S.Provisional Application No. 60/699,205 filed Jul. 14, 2005 (sometimesreferred to collectively as “related applications” herein).

BACKGROUND

1. Field of the Invention

This invention relates to a system and methods for creating and sharingan event using a social network. In particular, the system and methodshereof uses artificial reality to provide visual information to a userof the social network.

2. Description of the Related Art

U.S. Pat. Nos. 7,855,638; 8,275,397; 8,249,626; 8,207,843, and U.S.Publication No. 2008/0198230 (all incorporated herein by reference)relate generally to viewing people, places, and events, such as sportingevents, using positioning and artificial reality to improve the eventviewing experience. While a plan view of a map is useful for eventplanning, such views can be confusing and typically have limitedfunctions to interact with the event.

Commercial applications of augmented reality exist such as Layar,Wikitude, Junaio, Sekai Camera and others which use augmented reality toaid finding information about points of interest. See, e.g.,www.layar.com, www.wikitude.org/en/, and www.junaio.com.

Products or services that are tailored to the user are prevalent, suchas advertising models from Google based on search terms or advertisingbased on personal information of a user. For example, Apple postulatesdisplaying advertising to a mobile customer using one of its devicesbased on marketing factors. To compute marketing factors the Applesystem captures not only the machine identity, but search history,personal demographics, time of day, location, weather, loyalty programmembership, media library, user opinion or opinions of friends andfamily, etc. (collectively, referred to as “marketing factors”). See,e.g., U.S. Publication Nos. 2010/0125492; 2009/0175499; 2009/0017787;2009/0003662; 2009/0300122, and U.S. Pat. No. 7,933,900 (allincorporated herein by reference). Links to and use of social media,such as Facebook and Twitter, sometimes paired with location, are alsopossible indicators of a user behavior and user demographics. See e.g.,U.S. Publication Nos. 2009/0003662; 2011/0090252, and U.S. Pat. Nos.7,188,153; 7,117,254; 7,069,308 (all incorporated herein by reference).

Social networks are well known, and examples include LinkedIn.com,Google+ or Facebook.com and various social utilities to support socialnetworking. Growing a social network can mean that a person needs todiscover like-minded or compatible people who have similar interests orexperiences to him or her. Identifying like-minded people, however,often requires a substantial amount of and time and effort becauseidentifying new persons with common interests for friendships isdifficult. For example, when two strangers meet, it may take a long andawkward conversation to discover their common interests or experiences.

Social networks, in general, track and enable connections betweenmembers (including people, businesses, and other entities). Inparticular, social networking websites allow members to communicate moreefficiently information that is relevant to their friends or otherconnections in the social network. Social networks typically incorporatea system for maintaining connections among members in the social networkand links to content that is likely to be relevant to the members.Social networks also collect and maintain information about the membersof the social network. This information may be static, such asgeographic location, employer, job type, age, music preferences,interests, and a variety of other attributes, or it may be dynamic, suchas tracking a member's actions within the social network.

A typical modern computer-implemented social networking applicationrequires each user to provide some biographical information, and/oridentify his or her interests, and in some instances can suggest to theuser other users with compatible interests. For example, some web sitessuch as LinkedIn.com or Facebook.com require participants to register asmembers. Each member can fill out a profile or provide other personaldata such as professional interests, career information, interests inmusic, books, movies, and even information about political or religiousbeliefs. Matching algorithms can then use the profile or data providedto match members with members who are deemed compatible by thealgorithms, under the assumption, for example, that matching people'sinterests and values can lead to successful new friendships orrelationships within the social network. Some mobile device-basedapplications for identifying common interests require each user toconfigure the user's mobile device, including entering the user'sinterest, such as the things the user wishes to buy or sell, the kind ofpeople the user wishes to meet, etc., before a social networkingopportunity can be found for the user.

Typically, when a user who is also a member of a social network wishesto share information with other members of the social network, the usergenerally uploads or copies and pastes the information to a location onthe social network or forwards the information in the form of a messageor email to other members. Often, certain forms of information do notcopy and paste very well from one medium to another, so additionalformatting or modifications to the information may be required before itis suitable for viewing by other members. Therefore, the quality andtype of shared information is limited and members may be less likely toshare information with each other.

Various attempts have been made at sharing sports related eventinformation. For example, select information can be shared or publishedwith groups as more fully described in U.S. Pat. No. 7,478,078 and U.S.Publication Nos. 2008/0036653; 2011/0142016; 2010/0070758; 2011/0098928;and 2009/0144392. Related applications, including U.S. Pat. No.8,249,626 and U.S. Publication No. 2007/0117576 describe how locations,interest, social information, are created and published to groups orwithin defined groups. Groups can be formed based on certain criteria.Groups can be formed ahead of time or be dynamic. Social networkingparticipants can view shared views and AR messages.

SUMMARY OF THE INVENTION

Generally speaking, the system and methods of the present inventionenhance and expand the participant and spectator experience at an eventby sharing the event on a social network. The event includes views ofthe event from a participant position and AR messages overlaid the view.The participant(s) and spectators can communicate with each other,providing a mechanism for a remote “gallery” of spectators.

In one form, a method of sharing an event with members of a socialnetwork is provided and includes announcing to one or more members of asocial network an invitation to participate in said event and operatinga device accompanying said participant during the event to trackpositions of the participant during the event. The positions arecommunicated to a social media server associated with the socialnetwork. Some of the invited members subscribe to the event on thesocial network where they can view the event with a perspective viewfrom a participant position recorded on said social media server. Theview includes an augmented reality (AR) message and preferably a photobackground or artificial reality background.

In one embodiment, a system of sharing an event with members of a socialnetwork is provided and includes a device accompanying said participantduring the event and a server associated with said social network.

The system includes a communication link between the participant in theevent and a social network to announce to members of the social networkan opportunity to participate in said event. The device accompanies theparticipant during the event to track positions of said participantduring the event and a communication link between the device and theserver to communicate the positions to the social network server. Theserver operates to permit said members to become spectators to the eventon the social network and view the event where at least one view is aperspective view from a participant position recorded on the server andincludes an augmented reality (AR) message.

In one form, a method of compensating a participant in an event, isprovided and includes tracking the positions of the participant at theevent as the participant traverses the venue of the event. The positionsof the participant are communicated to members of a social network whohave elected to become spectators of the event. The event is viewed bythe spectators in real time where at least some of the views are aperspective view of the venue from a participant position and includesan AR message. The method includes a mechanism for compensating theparticipant by the spectators (or other participants) by rewardingcompensation to the participant based on a participant activity duringthe event, such as for golf, a birdie or a bet or a good round. Othersports have other goal oriented activities, such as winning a stage in abike race or fastest heat time in a ski slalom course. The compensationcan be in a variety of forms, including virtual goods, virtual services,awards, trophies, money, credits or donations to a charity. For example,in a professional or charity event spectators can donate money to adesignated charity based on outcomes or activities in the event.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a front elevation view of a smart phone having a graphicsdisplay;

FIG. 2 is a perspective view of a golf hole being played;

FIG. 3 is a front elevation view of the smart phone of FIG. 1 showingone representation of the golf hole of FIG. 2;

FIG. 4 is a front elevation view of the smart phone of FIG. 1 showinganother representation of the golf hole of FIG. 2;

FIG. 5 is a front elevation view of the smart phone of FIG. 1 showinganother representation of the golf hole of FIG. 2;

FIG. 6 is a block diagram depicting a wireless, client serverarchitecture in accordance with a preferred embodiment of the presentinvention;

FIG. 7 is a perspective view of a golf hole from a selected location,similar to FIG. 2;

FIG. 8 is a perspective of the back side the smart phone of FIG. 1;

FIG. 9 is a perspective of a portable device where the functionality isbuilt into glasses or goggles worn;

FIG. 10 is a perspective view of a ski event;

FIG. 11 is a perspective view of a bike event; and

FIG. 12 is a flow chart of an event.

DETAILED DESCRIPTION

High bandwidth, wireless networks are becoming commonplace, as is thecomputing power of mobile devices. Further rendering engines arebecoming readily available for wide ranging applications of artificialreality. Viewing an event, such as a sporting event, using a mobiledevice adds greatly to the user experience. Many sporting events, suchas golf, can be enhanced using a mobile device and artificial reality.U.S. Pat. No. 7,855,638 describes several examples of a system andmethod for viewing such events. In such event viewing systems, thebackground can be a real world image (e.g. photograph) or a virtualworld rendering, but in a preferred case, artificial reality is used toenhance the perspective viewing experience.

In creating such environments for the venue of the event, such as a golfcourse, bike race, ski competition, or the like, it is desirable toinsert virtual objects into the environment, such as messages, tips,targets, critiques, warnings, etc. The virtual objects overlay abackground environment, preferably as viewed from a user selectedposition. Typically, the user selected position is a participant presentposition as determined by GPS. Thus, in a preferred embodiment the sportparticipant (golfer, skier, biker, etc.) is presented with a perspectiveview of the event from the participant's current position (i.e.“viewpoint”) with virtual objects visually presented to the participant.

The present system and methods also address many sport related functionsthat can be used in such an artificial reality or mixed realityenvironment. For example, a basic function in golf is displaying inperspective view the golf hole from the golfer's location andidentifying distances to hazards or the green. However, other functionsexist, such as displaying suggested playing strategy based on thegolfer's personal data or profile, a similar handicap golfer, or assuggested by tips from a pro. Further, games can be incorporated intothe system and method such as corporate outing type experiences—closestto the pin, longest drive, accurate drive, as measured from anartificial reality target, etc. Other functions such as contests andbetting can also be incorporated.

In the present application, the term “message” is used to encompass anyartificial reality or virtual object, such as distance messages,advertisements, other facts, targets, shot distribution diagrams, eventinformation, warnings, announcements and other types of alpha numericdisplays. However, the message could also be a graphic, logo or brand.It shall be understood that other objects or graphics may also beenhanced and the term “message” is understood to include other objects.

In the present application, the term “social network” is used to referto any process or system that tracks and enables connections betweenmembers (including people, businesses, and other entities) or subsets ofmembers. The connections and membership may be static or dynamic and themembership can include various subsets within a social network. Forexample, a person's social network might include a subset of membersinterested in golf and the person shares a golf outing only with thegolf interest subset. Further, a social network might be dynamicallyconfigured. For example, a social network could be formed for “PebbleBeach” for August 3 and anyone interested could join the Pebble BeachAugust 3 social network. Alternatively, anyone within a certain range ofthe event might be permitted to join. The permutations involvingmembership in a social network are many and not intended to be limiting.

A social network that tracks and enables the interactive web by engagingusers to participate in, comment on and create content as a means ofcommunicating with their social graph, other users and the public. Inthe context of the present invention, such sharing and social networkparticipation includes participant created content and spectator createdcontent and of course, jointly created content. For example, the createdcontent can be interactive to allow spectators to add content to theparticipant created event.

Examples of conventional social networks include LinkedIn.com orFacebook.com, Google Plus, Twitter (including Tweetdeck), socialbrowsers such as Rockmelt, and various social utilities to supportsocial interactions including integrations with HTML5 browsers.www.Wikipedia.org/wiki/list_of_social_networking_sites lists severalhundred social networks in current use. Dating sites, Listservs, andInterest groups can also server as a social network. Interest groups orsubsets of a social network are particularly useful for inviting membersto attend an event, such as Google+ “circles” or Facebook “groups.”Individuals can build private social networks. Conventional socialnetworking websites allow members to communicate more efficientlyinformation that is relevant to their friends or other connections inthe social network. Social networks typically incorporate a system formaintaining connections among members in the social network and links tocontent that is likely to be relevant to the members. Social networksalso collect and maintain information about the members of the socialnetwork. This information may be static, such as geographic location,employer, job type, age, music preferences, interests, and a variety ofother attributes, or it may be dynamic, such as tracking a member'sactions within the social network. The methods and system hereof relateto dynamic events of a member's actions shared within a social network.

The most common positioning technology is GPS. As used herein,GPS—sometimes known as GNSS—is meant to include all of the current andfuture positioning systems that include satellites, such as the U.S.Navistar, GLONASS, Galileo, EGNOS, WAAS, MSAS, QZSS, etc. The accuracyof the positions, particularly of the participants, can be improvedusing known techniques, often called differential techniques, such asWAAS (wide area), LAAS (local area), Carrier-Phase Enhancement (CPGPS),Space Based Augmentation Systems (SBAS); Wide Area GPS Enhancement(WAGE), or Relative Kinematic Positioning (RKP). Even withoutdifferential correction, numerous improvements are increasing GPSaccuracy, such as the increase in the satellite constellation, multiplefrequencies (L₁, L₂, L₅), modeling and AGPS improvements, softwarereceivers, and ground station improvements. Of course, the positionaldegree of accuracy is driven by the requirements of the application. Inthe golf example used to illustrate a preferred embodiment, sub fivemeter accuracy provided by WAAS with Assisted GPS would normally beacceptable. Further, some “events” might be held indoors and the samemessage enhancement techniques described herein used. Such indoorpositioning systems include IMEO, Wi-Fi (Skyhook), Cell ID, pseudolites,repeaters, RSS on any electromagnetic signal (e.g. TV) and others knownor developed.

The term “geo-referenced” means a message fixed to a particular locationor object. Thus, the message might be fixed to a venue location, e.g.,golf course fence or fixed to a moving participant, e.g., a moving golfcar or player. An object is typically geo-referenced using either apositioning technology, such as GPS, but can also be geo-referencedusing machine vision. If machine vision is used (i.e. objectrecognition), applications can be “markerless” or use “markers,”sometimes known as “fiducials.” Marker-based augmented reality oftenuses a square marker with a high contrast. In this case, four cornerpoints of a square are detected by machine vision using the squaremarker and three-dimensional camera information is computed using thisinformation. Other detectable sources have also been used, such asembedded LED's or special coatings or QR codes. Applying AR to a markerwhich is easily detected is advantageous in that recognition andtracking are relatively accurate, even if performed in real time. So, inapplications where precise registration of the AR message in thebackground environment is important, a marker based system has someadvantages.

In a “markerless” system, AR uses a general natural image instead of afiducial. In general, markerless AR uses a feature point matchingmethod. Feature point matching refers to an operation for searching forand connecting the same feature points in two different images. A methodfor extracting a plane using a Simultaneous Localization andMap-building (SLAM)/Parallel Tracking and Mapping (PTAM) algorithm fortracking three-dimensional positional information of a camera andthree-dimensional positional information of feature points in real timeand providing AR using the plane has been suggested. However, since theSLAM/PTAM algorithm acquires the image so as to search for the featurepoints, computes the three-dimensional position of the camera and thethree-dimensional positions of the feature points, and provides AR basedon such information, a considerable computation is necessary. A hybridsystem can also be used where a readily recognized symbol or brand isgeo-referenced and machine vision substitutes the AR message.

In the present application, the venue for the sporting event can be areal view or depicted as a photo background environment or a virtualenvironment, or a mixture, sometimes referred to as “mixed reality.” Aconvenient way of understanding the messages of the present invention isas a layer of artificial reality or “augmented reality” messagesoverlaid the sporting event venue background. There are differentmethods of creating the sporting event venue background as understood byone of ordinary skill in the art. For example, an artificial backgroundenvironment can be created by a number of rendering engines, sometimesknown as a “virtual” environment. See, e.g., Nokia's (through its Navteqsubsidiary) Journey View which blends digital images of a realenvironment with an artificial 3D rendering. A real environment can bethe background as seen through glasses of FIG. 9, but can also becreated using a digital image. Such a digital image can be stored andretrieved for use, such as a “street view” or photo, video, or panorama,or other type of stored image. Alternatively, many mobile devices have acamera for capturing a digital image which can be used as the backgroundenvironment. Such a camera-sourced digital image may come from the user,friends, social network groups, crowd-sourced, or service provided.Because the use of a real environment as the background is common,“augmented reality” (AR) often refers to a technology of inserting avirtual reality graphic (object) into an actual digital image andgenerating an image in which a real object and a virtual object aremixed (i.e. “mixed reality”). AR is characterized in that supplementaryinformation using a virtual graphic may be layered or provided onto animage acquired of the real world. Multiple layers of real and virtualreality can be mixed. In such applications the placement of an object or“registration” with other layers is important. That is, the position ofobjects or layers relative to each other based on a positioning systemshould be close enough to support the application. As used herein,“artificial reality” (“AR”) is sometimes used interchangeably with“mixed” or “augmented” reality, it being understood that the backgroundenvironment can be real or virtual.

I. Overview

In the drawings, golf is used as an example of an event where the eventcan be created and shared on a social network, enhancing and expandingthe participant and spectator experience. Turning to the drawings, anillustrative embodiment uses a mobile device, such as smart phone 10 ofFIG. 1, accompanying the golfer. The golfer selects AR application 106on the touch sensitive graphics display 102. Smart phone 10 includes avariety of sensors, including a GPS unit for determining its location, agyroscope for determining the orientation, an accelerometer, ambientlight sensor and a digital compass. Additionally, phone 10 includes oneor more radios, such as a packet radio, a cell radio, WiFi, Bluetooth,and near field. Of course, other devices can be used such as thededicated golf handheld devices as well as a tablet computer having GPS,especially the tablets with screen sizes larger than a smart phone butsmaller than about 10 inches to aid portability, such as a Dell Streak,Motorola Xoom, or Samsung Galaxy. In some embodiments, the device can bea tablet affixed to a golf cart with a camera oriented in the directionof travel. That is, in some embodiments, a wireless camera connected toa Bluetooth compatible device 10 may be preferred. Examples of suchcameras are JonesCAM LX, Vuzix iWear CamAR available from VuzixCorporation, Rochester, N.Y., AT-1Wireless available from Dogcam, andContourGPS available from Contour HD.

FIG. 2 is an illustrative example of golf hole 90 used to describeaspects of the present invention. In this case, player 80 is shooting togreen 82 and accompanied by his golf bag 84 while holding device 10oriented towards green 82. Lake 85 presents a significant hazard to thegolfer, as does green side bunker 83. For illustrative purposes, in FIG.2 the pin on green 82 is 232 yards from the golfer's current location81, while the near side of lake 85 is 175 yards, and the far side oflake 85 or carry distance on a line between the current location and thepin is 194 yards.

FIG. 3 illustrates the perspective view of golf hole 90 of FIG. 2 asseen on smart phone 10 from the golfer's current location 81. As can beseen, artificial reality message 92 shows that 194 yards is necessary tocarry lake 85. Further, AR message 94 shows that the pin is 232 yardsfrom the golfer's current location 81. Further, a pro tip AR message 96gives the golfer some insight into the green slope. The AR messages canbe color coded if desired, e.g., blue for distance and red marking aboundary or hazard.

In a preferred embodiment, smart phone 10 is held in the golfer's handis vertically oriented and positioned in the direction of the desiredshot, such as from golfer 80 to green 82 as shown in FIG. 2. Thebackground of the perspective view of golf hole 90 on display 102 isfrom the GPS determined position of phone 10 using backside camera 141(FIG. 9). In a preferred embodiment, backside camera 141 is used inphone 10 to provide a photo background from the camera. The AR messagesare overlaid the background as seen in FIG. 3. In an alternativeembodiment, a virtual background is used and player 80 can more easilyselect another location, such as behind the golfer or from the green tothe golfer for viewing the play of a golf round. The ability to select astarting location or viewpoint is particularly useful when golfer 80 isreplaying his round at a later date on a home computing device.

In FIG. 4, AR target area message 120 is illustrated. In thisembodiment, golfer 80 is an average handicap golfer whose personalprofile shows a shot distribution of a 7 Iron as 145 yards to 165 yardswith a 20 yard diameter (i.e. 155 yards with a 20 yard CEP). Asdiscussed below, shot distribution can be highly customized to includeactual club use, as adjusted by current winds, density altitude,equipment used, and elevation, or can be a normalized distributionselected by the golfer for a particular skill level. In any event, thedevice 10 shows target 120 and suggested club for use in window 122.Note that window 98 is used to always show distance to pin (or anapproximation such as center of green). In the case illustrated in FIG.4, the suggestion is to lay up short and to the right of lake 85 using a7 iron.

FIG. 5 is similar to FIG. 4, but in this case golfer 80 has an aboveaverage personal profile. In this case, the personal profile shows ashot distribution of a 3 Iron as 195 yards to 225 yards with a 30 yarddiameter (i.e. 210 yards with a 30 yard CEP). In the case illustrated inFIG. 5, the suggestion is to carry lake 85 aiming for the left side ofgreen 82 aiming for target area 124 using a 3 iron.

FIG. 6 illustrates network 40 for the golf course which is useful insome embodiments of the methods and systems of the present invention. Anumber of participants (golfers) 41 are equipped with a positioningmechanism, such as GPS, with position and other information transmittedby radio to cell tower 42 connected to server 44. Cell tower 42 can bepart of a conventional cell network or be an independent picocell orfemtocell as is known in the art. The GPS derived position of eachparticipant (golfer) 41 can be corrected and accuracy improved ifdesired, but is usually less than 5 meters. The participant (golfer)positions, AR messages, and other information is stored by server 44(e.g. located in the club house) and can be transmitted by radio 46 toany or all other participants or spectators 48 if desired. Groups can beformed or pre-established and information distributed to one or moregroups. That is, each participant/spectator 48 has a smart phone 10 forreceiving the transmitted participant positions. Of course, server 44can also transmit golfer position information and communications to aclub house, remote, or home spectators via Internet connection 49. Suchhome or club house spectators can, if desired, call up a screen on theirTV or alternatively, watch the golf match on a home computer or otherdevice.

In a preferred form, the participant 41 creates an “event” for a socialmedia which is published to a selected social network. The individualshaving access to the event can be selected ad hoc or established priorto the event. The social group having access would typically includeother participants, as well as spectators 48. As shown in FIG. 6, thenetwork allows sharing of event information among the selected socialgroup, and includes not only the position of one or more participants atthe event venue, but also participant sensor information and playinformation, and communications among the participants 41 and spectators48. The sharing of the event can be real time or replayed after theevent.

II. Mobile Device

In more detail, FIG. 1 is a front elevational view of a smart phone ormobile device 10, which is the preferred form factor for the devicediscussed herein to illustrate certain aspects of the present invention.Mobile device 10 can be, for example, a handheld computer, a tabletcomputer, a personal digital assistant, a cellular telephone, a camerahaving a GPS and a radio, a GPS with a radio, a network appliance, acamera, a smart phone, an enhanced general packet radio service (EGPRS)mobile phone, a network base station, a media player, a navigationdevice, an email device, a game console, or other electronic device or acombination of any two or more of these data processing devices or otherdata processing.

Mobile device 10 includes a touch-sensitive graphics display 102. Thetouch-sensitive display 102 can implement liquid crystal display (LCD)technology, light emitting polymer display (LPD) technology, or someother display technology. The touch-sensitive display 102 can besensitive to haptic and/or tactile contact with a user.

The touch-sensitive graphics display 102 can comprise amulti-touch-sensitive display. A multi-touch-sensitive display 102 can,for example, process multiple simultaneous touch points, includingprocessing data related to the pressure, degree and/or position of eachtouch point. Such processing facilitates gestures and interactions withmultiple fingers, chording, and other interactions. Othertouch-sensitive display technologies can also be used, e.g., a displayin which contact is made using a stylus or other pointing device. Anexample of a multi-touch-sensitive display technology is described inU.S. Pat. Nos. 6,323,846; 6,570,557; 6,677,932; and U.S. Publication No.2002/0015024, each of which is incorporated by reference herein in itsentirety. Touch screen 102 and touch screen controller can, for example,detect contact and movement or break thereof using any of a plurality oftouch sensitivity technologies, including but not limited to capacitive,resistive, infrared, and surface acoustic wave technologies, as well asother proximity sensor arrays or other elements for determining one ormore points of contact with touch screen 102.

Mobile device 10 can display one or more graphical user interfaces onthe touch-sensitive display 102 for providing the user access to varioussystem objects and for conveying information to the user. The graphicaluser interface can include one or more display objects 104, 106. Each ofthe display objects 104, 106 can be a graphic representation of a systemobject. Some examples of system objects include device functions,applications, windows, files, alerts, events, or other identifiablesystem objects.

Mobile device 10 can implement multiple device functionalities, such asa telephony device, as indicated by a phone object; an e-mail device, asindicated by the e-mail object; a network data communication device, asindicated by the Web object; a Wi-Fi base station device (not shown);and a media processing device, as indicated by the media player object.For convenience, the device objects, e.g., the phone object, the e-mailobject, the Web object, and the media player object, can be displayed inmenu bar 118.

Each of the device functionalities can be accessed from a top-levelgraphical user interface, such as the graphical user interfaceillustrated in FIG. 1. Touching one of the objects e.g. 104, 106, etc.can, for example, invoke the corresponding functionality. In theillustrated embodiment, object 106 represents an Artificial Realityapplication in accordance with the present invention.

Upon invocation of particular device functionality, the graphical userinterface of mobile device 10 changes, or is augmented or replaced withanother user interface or user interface elements, to facilitate useraccess to particular functions associated with the corresponding devicefunctionality. For example, in response to a user touching the phoneobject, the graphical user interface of the touch-sensitive display 102may present display objects related to various phone functions;likewise, touching of the email object may cause the graphical userinterface to present display objects related to various e-mailfunctions; touching the Web object may cause the graphical userinterface to present display objects related to various Web-surfingfunctions; and touching the media player object may cause the graphicaluser interface to present display objects related to various mediaprocessing functions.

The top-level graphical user interface environment or state of FIG. 1can be restored by pressing button 120 located near the bottom of mobiledevice 10. Each corresponding device functionality may havecorresponding “home” display objects displayed on the touch-sensitivedisplay 102, and the graphical user interface environment of FIG. 1 canbe restored by pressing the “home” display object.

The top-level graphical user interface is shown in FIG. 1 and caninclude additional display objects, such as a short messaging service(SMS) object, a calendar object, a photos object, a camera object, acalculator object, a stocks object, a weather object, a maps object, anotes object, a clock object, an address book object, and a settingsobject, as well as AR object 106. Touching the SMS display object can,for example, invoke an SMS messaging environment and supportingfunctionality. Likewise, each selection of a display object can invoke acorresponding object environment and functionality.

Mobile device 10 can include one or more input/output (I/O) devicesand/or sensor devices. For example, speaker 122 and microphone 124 canbe included to facilitate voice-enabled functionalities, such as phoneand voice mail functions. In some implementations, loud speaker 122 canbe included to facilitate hands-free voice functionalities, such asspeaker phone functions. An audio jack can also be included for use ofheadphones and/or a microphone.

A proximity sensor (not shown) can be included to facilitate thedetection of the user positioning mobile device 10 proximate to theuser's ear and, in response, disengage the touch-sensitive display 102to prevent accidental function invocations. In some implementations, thetouch-sensitive display 102 can be turned off to conserve additionalpower when mobile device 10 is proximate to the user's ear.

Other sensors can also be used. For example, an ambient light sensor(not shown) can be utilized to facilitate adjusting the brightness ofthe touch-sensitive display 102. An accelerometer (not shown) can beutilized to detect movement of mobile device 10, as indicated by thedirectional arrow. Accordingly, display objects and/or media can bepresented according to a detected orientation, e.g., portrait orlandscape.

Mobile device 10 may include circuitry and sensors for supporting alocation determining capability, such as that provided by the globalpositioning system (GPS) or other positioning system (e.g., Cell ID,systems using Wi-Fi access points, television signals, cellular grids,Uniform Resource Locators (URLs)). A positioning system (e.g., a GPSreceiver) can be integrated into the mobile device 10 or provided as aseparate device that can be coupled to the mobile device 10 through aninterface (e.g., port device 132) to provide access to location-basedservices.

Mobile device 10 can also include a front camera lens and sensor 140. Ina preferred implementation, a backside camera lens and sensor 141 islocated on the back surface of the mobile device 10 as shown in FIG. 9.The cameras 140, 141 can capture still images and/or video. The camerasubsystems and optical sensors 140, 141 may comprise, e.g., a chargedcoupled device (CCD) or a complementary metal-oxide semiconductor (CMOS)optical sensor, can be utilized to facilitate camera functions, such asrecording photographs and video clips. Camera controls (zoom, pan,capture and store) can be incorporated into buttons 134-136 (FIG. 1.)

The preferred mobile device 10 includes a GPS positioning system. Inthis configuration, another positioning system can be provided by aseparate device coupled to the mobile device 10, or can be providedinternal to the mobile device. Such a positioning system can employpositioning technology including a GPS, a cellular grid, URL's, IMEO,pseudolites, repeaters, Wi-Fi or any other technology for determiningthe geographic location of a device. The positioning system can employ aservice provided by a positioning service such as, for example, a Wi-FiRSS system from SkyHook Wireless of Boston, Mass., or Rosum Corporationof Mountain View, Calif. In other implementations, the positioningsystem can be provided by an accelerometer and a compass using deadreckoning techniques starting from a known (e.g. determined by GPS)location. In such implementations, the user can occasionally reset thepositioning system by marking the mobile device's presence at a knownlocation (e.g., a landmark or intersection). In still otherimplementations, the user can enter a set of position coordinates (e.g.,latitude, longitude) for the mobile device. For example, the positioncoordinates can be typed into the phone (e.g., using a virtual keyboard)or selected by touching a point on a map. Position coordinates can alsobe acquired from another device (e.g., a car navigation system) bysyncing or linking with the other device. In other implementations, thepositioning system can be provided by using wireless signal strength andone or more locations of known wireless signal sources (Wi-Fi, TV, FM)to provide the current location. Wireless signal sources can includeaccess points and/or cellular towers. Other techniques to determine acurrent location of the mobile device 10 can be used and otherconfigurations of the positioning system are possible.

Mobile device 10 can also include one or more wireless communicationsubsystems, such as a 802.11b/g/n communication device, and/or aBluetooth™ communication device, in addition to near fieldcommunications. Other communication protocols can also be supported,including other 802.x communication protocols (e.g., WiMax, Wi-Fi), codedivision multiple access (CDMA), global system for mobile communications(GSM), Enhanced Data GSM Environment (EDGE), 3G (e.g., EV-DO, UMTS,HSDPA), etc. Additional sensors are incorporated into the device 10,such as accelerometer, digital compass and gyroscope. Further,peripheral sensors, devices and subsystems can be coupled to peripheralsinterface 132 to facilitate multiple functionalities. For example, amotion sensor, a light sensor, and a proximity sensor can be coupled toperipherals interface 132 to facilitate the orientation, lighting andproximity functions described with respect to FIG. 1. Other sensors canalso be connected to peripherals interface 132, such as a GPS receiver,a temperature sensor, a biometric sensor, or other sensing device, tofacilitate related functionalities.

Port device 132, is e.g., a Universal Serial Bus (USB) port, or adocking port, or some other wired port connection. Port device 132 can,for example, be utilized to establish a wired connection to othercomputing devices, such as other communication devices 10, a personalcomputer, a printer, or other processing devices capable of receivingand/or transmitting data. In some implementations, port device 132allows mobile device 10 to synchronize with a host device using one ormore protocols.

Input/output and operational buttons are shown at 134-136 to control theoperation of device 10 in addition to, or in lieu of the touch sensitivescreen 102. Mobile device 10 can include a memory interface to one ormore data processors, image processors and/or central processing units,and a peripherals interface. The memory interface, the one or moreprocessors and/or the peripherals interface can be separate componentsor can be integrated in one or more integrated circuits. The variouscomponents in mobile device 10 can be coupled by one or morecommunication buses or signal lines.

Preferably, the mobile device includes a graphics processing unit (GPU)coupled to the CPU. While a Nvidia GeForce GPU is preferred, in partbecause of the availability of CUDA, any GPU compatible with OpenGL isacceptable. Tools available from Kronos allow for rapid development of3D models.

The I/O subsystem can include a touch screen controller and/or otherinput controller(s). The touch-screen controller can be coupled to touchscreen 102. The other input controller(s) can be coupled to otherinput/control devices 132-136, such as one or more buttons, rockerswitches, thumb-wheel, infrared port, USB port, and/or a pointer devicesuch as a stylus. The one or more buttons (132-136) can include anup/down button for volume control of speaker 122 and/or microphone 124,or to control operation of cameras 140, 141. Further, the buttons(132-136) can be used to “capture” and share an image of the event alongwith the location of the image capture.

In one implementation, a pressing of button 136 for a first duration maydisengage a lock of touch screen 102; and a pressing of the button for asecond duration that is longer than the first duration may turn thepower on or off to mobile device 10. The user may be able to customize afunctionality of one or more of the buttons. Touch screen 102 can, forexample, also be used to implement virtual or soft buttons and/or akeyboard.

In some implementations, mobile device 10 can present recorded audioand/or video files, such as MP3, AAC, and MPEG files. In someimplementations, mobile device 10 can include the functionality of anMP3 player, such as an iPod™. Mobile device 10 may, therefore, include a36-pin connector that is compatible with the iPod. Other input/outputand control devices can also be used.

The memory interface can be coupled to a memory. The memory can includehigh-speed random access memory and/or non-volatile memory, such as oneor more magnetic disk storage devices, one or more optical storagedevices, and/or flash memory (e.g., NAND, NOR). The memory can store anoperating system, such as Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, oran embedded operating system such as VxWorks. The operating system mayinclude instructions for handling basic system services and forperforming hardware dependent tasks. In some implementations, theoperating system handles timekeeping tasks, including maintaining thedate and time (e.g., a clock) on the mobile device 10. In someimplementations, the operating system can be a kernel (e.g., UNIXkernel).

The memory may also store communication instructions to facilitatecommunicating with one or more additional devices, one or more computersand/or one or more servers. The memory may include graphical userinterface instructions to facilitate graphic user interface processing;sensor processing instructions to facilitate sensor-related processingand functions; phone instructions to facilitate phone-related processesand functions; electronic messaging instructions to facilitateelectronic-messaging related processes and functions; web browsinginstructions to facilitate web browsing-related processes and functions;media processing instructions to facilitate media processing-relatedprocesses and functions; GPS/Navigation instructions to facilitate GPSand navigation-related processes and instructions; camera instructionsto facilitate camera-related processes and functions; other softwareinstructions to facilitate other related processes and functions; and/ordiagnostic instructions to facilitate diagnostic processes andfunctions. The memory can also store data, including but not limited tocoarse information, locations (points of interest), personal profile,documents, images, video files, audio files, and other data. Theinformation can be stored and accessed using known methods, such as astructured or relative database.

Portable device 220 of FIG. 8 is an alternative embodiment in theconfiguration of glasses or goggles and includes a GPS and patch antenna232, microprocessor 234, and radio 236. Controls, such as thedirectional pad 224, are on the side frames (opposite side not shown).Batteries are stored in compartment 242. The displays are transparentLCD's as at 244. Examples of such a device are the MyVue headset made byMicroOptical Corp. of Westwood, Mass. (see, U.S. Pat. No. 6,879,443),Vuzix Wrap 920 AR, 1200 VR, and Tac-Eye LT available from VuzixCorporation, Rochester, N.Y. A particular benefit of the use of wearableglasses such as the embodiment of FIG. 8 is the ability to incorporateaugmented reality messages, e.g. point of interest overlays onto the“real” background. In the golf example, a golfer wearing glasses 220 cansee the AR messages and selectively highlight a particular message andadditional information relative to that message (e.g. wind used in clubselection, next best club selection, status of other golfers rounds,etc.). See, e.g. U.S. Pat. Nos. 7,002,551; 6,919,867; 7,046,214;6,945,869; 6,903,752; 6,317,127 (herein incorporated by reference).

III. Network Operating Environment

In FIG. 6, a depiction of real-time network 40 useful in manyembodiments is shown. It should be understood that in many uses areal-time network environment as illustrated in FIG. 6 is not necessary.That is, information concerning an event can alternatively be recordedand uploaded to a social network server after the event. In thereal-time embodiment of FIG. 6 participants/golfers 41 communicate withcell base station 42 preferably using the cellular network which caninclude one or more femtocells or picocells. While simple data can betransmitted on the control plane (e.g. GPRS) preferably the cell radiouses a data plan, i.e. the user plane. The location, communication, andother data is communicated between golfers 41 and social media server44. Server 44 stores the position data of each golfer 41 communicated tocell base station 42, and other pertinent data such as golfer shotposition, scores, etc. Such other data can, in addition to sensor dataderived from device 10, comprise sensor data from the participant, suchas from instrumented golf clubs. See, e.g., U.S. Publication Nos.2011/0143848 and 2008/0051208 (incorporated by reference). In apreferred form, server 44 stores the points of interest or coursedatabase which is used to create many of the AR messages.

Other participants/golfers 41 can send messages to selected golfers orgroups of golfers which are stored on server 44. For example, a location(green or feature) of a golf course can be “tagged” with an AR messageto other golfers (e.g. “Joe hit OB here”). Of course, server 44 canstore advertising messages as well for delivery to golfers, such asspecials in the pro shop or “deal of the day” such as living social orGroupon opportunities. Server 44 can also be used for authentication ofgraphic devices 10 and enable selectable purchases from golfers orspectators (i.e. refreshments or for delivery or pickup at the turn, orballs, deal of day, etc). Server 44 can also process the incomingposition data to increase the accuracy if desired. For example, server44 can include its own base station GPS and apply a correction to agolfer's position if desired. However, in the embodiment of FIG. 6 usingthe cellular network, Assisted GPS using mobile station assistance isavailable and used, thus improving time to first fix, sensitivity andaccuracy, so additional accuracy is not normally deemed necessary.

Radio 46 is used to communicate among the golfers 41 and spectators 48.For spectators in attendance, the cell network is preferably used. 4Gcellular networks such as LTE, or Long Term Evolution, have downloadspeeds (e.g. 12 mbps) surpassing WiFi and may become acceptablesubstitutes. For example, WiMax (Sprint >10 mbps); LTE (Verizon 40-50mbps) (AT&T unknown); and HSPA+ (T mobile 21 mbps) (AT&T 16 mbps) appearacceptable 4G network speeds. In many cases, with high performance 4Gcellular networks, the social media server 44 need not be local, i.e.proximate to the golf course. However, if a cell network is not used,the radio 46 of network of FIG. 6 can be local, i.e. a WiFi or 900 Mhzlocal area network is used. In this case radio 46 preferably uses WiFi(802.11 b/g/n) to transmit to golfers/spectators 48.

Special requests from golfers or spectators 48 can be made to server 44,such as for streaming video of a particular scene (e.g. a “flyover” ofthe hole), refreshment orders, memorabilia purchases, etc. This functionis shown at 50, 52 in FIG. 6.

Some spectators 48 may be remote from the sporting event. In this case,server 44 can transmit the desired information over internet connection49 to the club house, home computer or television remote from the event.While one embodiment has been described in the context of a spectator inphysical attendance at the golf course with information broadcast byradio, the use of device 10 at remote locations is equally feasible. Inanother embodiment more suited for remote locations, for example,portable device 10 can be used at home while watching a golf event onTV, with the participant location and other information streaming overthe internet. WiFi in the home is a preferred mode of broadcasting theinformation between the portable device and the network.

One function of the server 44 is to allow observation of a round by agolfer, either in real time or post play. That is, the views of FIGS.2-5 can be posted to the server 44 and observed by a spectator 48 usingany graphic device, including a personal computer, tablet, or a cellphone. Similar to using graphic device 10 coupled to the internet, apersonal computer spectator can select the source or position oforigination of the desired view, and the target or orientation from thesource or target. Elevations, zoom, pan, tilt, etc. may be selected bythe remote spectator as desired to change the origin viewpoint or size.

In “my view,” for example, the remote location graphic device mightdisplay only information from the golfer's shot location 81.Alternatively, the spectator might want a selectable view, such asoverhead in plan form, from behind the golfer (FIG. 7) or other locationsuch as from the pin to the golfer's location. In any of these modes,the remote location spectator could zoom, pan or tilt as describedabove, freeze, slow motion, replay, etc. to obtain a selected view onthe portable device 10.

While the preferred embodiment contemplates most processing occurring atdevice 10, different amounts of preprocessing of the position data canbe processed at server 44. For example, the participant information canbe differentially corrected at the server (e.g. in addition to WAAS or alocal area differential correction) or at device 10 or even informationpost-processed with carrier phase differential to achieve centimeteraccuracy. Further, it is anticipated that most of the graphics renderingcan be accomplished at portable device 10, but an engineering choicewould be to preprocesses some of the location and rendering informationat server 44 prior to broadcast. In particular, many smart phones andhandheld computers include GPU's which enable photorealistic renderingand the developers have access to advanced tools for development such asOpenGL and CUDA.

Mobile device 10 of FIGS. 1 and 9 preferably accompanies some of golfers41 of FIG. 6 in attendance at the course. Devices 10 communicate overone or more wired and/or wireless networks 46 in data communication withserver 44. In addition, the devices can communicate with a wirelessnetwork, e.g., a cellular network, or communicate with a wide areanetwork (WAN), such as the Internet, by use of a gateway. Likewise, anaccess point associated with Radio 46, such as an 802.11b/g/n wirelessaccess point, can provide communication access to a wide area network.

Both voice and data communications can be established over the wirelessnetwork of FIG. 6 and access point 46 or using a cellular network. Forexample, mobile device 10 a can place and receive phone calls (e.g.,using VoIP protocols), send and receive e-mail messages (e.g., usingPOP3 protocol), and retrieve electronic documents and/or streams, suchas web pages, photographs, and videos, over the wireless network,gateway, and wide area network (e.g., using TCP/IP or UDP protocols).Likewise, mobile device 10 can place and receive phone calls, send andreceive e-mail messages, and retrieve electronic documents over accesspoint 46 and the wide area network. In some implementations, mobiledevice 10 can be physically connected to access point 46 using one ormore cables and the access point can be a personal computer. In thisconfiguration, mobile device 10 can be referred to as a “tethered”device.

Mobile devices 10 can also establish communications by other means. Forexample, wireless device 10 can communicate with other wireless devices,e.g., other wireless devices 10, cell phones, etc., over a wirelessnetwork. Likewise, mobile devices 10 can establish peer-to-peercommunications, e.g., a personal area network, by use of one or morecommunication subsystems, such as the Bluetooth™ communication device.Other communication protocols and topologies can also be implemented.

In use in the play of golf, it is believed preferable to use a realenvironment as the background, such as a digital image captured bybackside camera 141 of FIG. 9. In many cases, this real backgroundenvironment can be augmented with other golfers pictures or archivedvideo. Typically, the golfer 41 would activate one of the buttons134-136 at each shot so that the spectators 48 could more easily followthe round.

In other embodiments, a virtual environment may be used as thebackground. In such cases, server 44 preferably uses the OTOY, Gaikai,or OnLive video compression technology to transmit the participantposition information the virtual background environment, as well as theAR objects, such as each car 54. OTOY (and Gaikai and OnLive) are cloudbased gaming and application vendors that can transmit real timephotorealistic gaming to remote gamers. Such companies that renderphotorealistic 3D games for realtime remote play are Otoy, see, e.g.,www.otoy.com; OnLive, see, e.g., en.wikipedia.org/wiki/ OnLive; andGaikai, see, e.g.,technabob.com/blog/2010/03/16/gaikai-cloud-based-gaming. Onlive, forexample, advertises that with 5 mbps it can transfer 220 frames persecond with 12-17 ms latency, employed advanced graphics—ajax, flash,Java, ActiveX.

The goal is high performance game systems that are hardware and softwareagnostic. That is, a goal is intense game processing performed on aremote server and communicated to the remote user. Using such cloudbased gaming technology, smart phones 10 can run any of the advancedbrowsers (e.g. IE9 or Chrome) running HTML5 that support 3D graphics.However, other AR specific browsers can alternatively be used, such asavailable from Layar, Junaio, Wikitude, Sekai Camera or Mixare(www.mixare.org). While OTOY (and Gaikai and OnLive) promise nodiscernable latency in their gaming environment, server 44 for the golfevent of FIG. 6 is preferably placed at the venue of the event.

Therefore, the amount of processing occurring at server 44 versus device10 is a design choice based on the event, the background, the radionetwork available, the computational and display capability available atdevice 10 or other factors.

In addition the content of the advertisement messages can be changedbased on context. Such smart phones 10 have not only machine ID's, butalso search history, location history, and even personal information.Further, the user might be identified based on social mediaparticipation—e.g. Facebook or Twitter accounts. Such information isconsidered “context” in the present application, along with the typicaldemographics of an event and “marketing factors” as previouslydiscussed. That is, the event might have its own context which indicatesthe demographic profile of most of the spectators at the event. A golfmatch might have a context of golf spectators with adequate disposableincome to purchase a vehicle. Therefore, advertising a Buick as shown inFIG. 7 makes advertising sense. See, U.S. patent application Ser. No.13/152,476 (incorporated by reference).

In a preferred embodiment, a golfer 41 would “capture” his round usingthe camera function and including his GPS coordinates and other sensordata (such as club selection) as described above. The golfer 41 wouldpost his round in real-time to the social media server 44. Using asocial media relationship, a golfer 41 would “host” a round and provideaccess to spectators 48, who might be selected followers or friends onthe social media site. Preferably, the golfer 41 is the “host” of theevent and the spectators 48 are the gallery. The spectators 48 cancomment during play of the round such as providing instruction orcritique, or even betting on outcomes with the golfer 41 during play orpresenting challenges to the golfer 41.

The round can also be used in addition to, or in lieu of, the real-timeround. That is, the golfer 41 can post a round to a social media server44 after play and notify his followers/spectators 48 of the commencementof the round. During replay, prior comments can be observed and newcomments posted. In this way, the golfer 41 is sharing his round withspectators 48 either in real-time or historically. Advantageously, anumber of golfers 41 can participate in the same round, showing theiroutcomes and comments to the spectators 48.

While the play of golf has been used to illustrate the use of thenetwork 40 of FIG. 6, it should be understood that the event is notlimited to a round of golf. That is, many sporting events can be postedto a social media server 44 for review and comment. Biking, car racing,motocross (or any motorized sports), running, sailing, skiing, boating,as well as non-sport related activities can be shared via the socialmedia server 44. For example, the user 41 of FIG. 6 could be a hiker onthe Appalachian trail sharing the hike (“event”) with remote spectators48 in real-time and after the hike. The event is not limited to amateursports and can include professional sports as well.

IV. Graphics

The graphics generated on screen 102 can be 2D graphics, such asgeometric models (also called vector graphics) or digital images (alsocalled raster graphics). In 2D graphics, these components can bemodified and manipulated by two-dimensional geometric transformationssuch as translation, rotation, scaling. In object oriented graphics, theimage is described indirectly by an object endowed with a self-renderingmethod—a procedure which assigns colors to the image pixels by anarbitrary algorithm. Complex models can be built by combining simplerobjects, in the paradigms of object-oriented programming. Moderncomputer graphics card displays almost overwhelmingly use rastertechniques, dividing the screen into a rectangular grid of pixels, dueto the relatively low cost of raster-based video hardware as comparedwith vector graphic hardware. Most graphic hardware has internal supportfor blitting operations and sprite drawing.

Preferably, however, the graphics generated on screen 102 are 3D. OpenGLand Direct3D are two popular APIs for the generation of real-timeimagery in 3D. Real-time means that image generation occurs in “realtime” or “on the fly”). Many modern graphics cards provide some degreeof hardware acceleration based on these APIs, frequently enabling thedisplay of complex 3D graphics in real-time. However, it's not necessaryto employ any one of these to actually create 3D imagery. The graphicspipeline technology is advancing dramatically, mainly driven by gamingapplications enabling more realistic 3D synthetic renderings of FIGS.1-5.

3D graphics have become so popular, particularly in computer games, thatspecialized APIs (application programmer interfaces) have been createdto ease the processes in all stages of computer graphics generation.These APIs have also proved vital to computer graphics hardwaremanufacturers, as they provide a way for programmers to access thehardware in an abstract way, while still taking advantage of the specialhardware of this-or-that graphics card.

These APIs for 3D computer graphics are particularly popular:

OpenGL and the OpenGL Shading Language

OpenGL ES 3D API for embedded devices

Direct3D (a subset of DirectX)

RenderMan

RenderWare

Glide API

TruDimension LC Glasses and 3D monitor API

OpenGL is widely used and many tools are available from firms such asKronos. There are also higher-level 3D scene-graph APIs which provideadditional functionality on top of the lower-level rendering API. Suchlibraries under active development include:

QSDK

Quesa

Java 3D

JSR 184 (M3G)

NVidia Scene Graph

OpenSceneGraph

OpenSG

OGRE

Irrlicht

Hoops3D

Photo-realistic image quality is often the desired outcome, and to thisend several different, and often specialized, rendering methods havebeen developed. These methods range from the distinctly non-realisticwireframe rendering through polygon-based rendering, to more advancedtechniques such as: scanline rendering, ray tracing, or radiosity. Therendering process is computationally expensive, given the complexvariety of physical processes being simulated. Computer processing powerhas increased rapidly over the years, allowing for a progressivelyhigher degree of realistic rendering. Film studios that producecomputer-generated animations typically make use of a render farm togenerate images in a timely manner. However, falling hardware costs meanthat it is entirely possible to create small amounts of 3D animation ona small processor, such as in the device 10. Driven by the game studios,hardware manufacturers such as ATI, Nvidia, Creative Labs, and Ageiahave developed graphics accelerators which greatly increase the 3Drendering capability. It can be anticipated that in the future, one ormore graphics rendering chips, such as the Ageia Physx chip, or theGeForce GPU's will enable full rendering at the device 10.

While full 3D photorealistic rendering is difficult with the device 10described herein standing alone, advances in processing and renderingcapability will enable greater use of 3D graphics in the future. In thegolf application, a golfer object and a course object (e.g., Doral orAugusta National) can be rendered in advance and stored, makingrealistic 3D graphics possible. However, a preferred form is to use acloud-based gaming provider, such as OTOY, OnLive, or Gaikai at server44 networked to devices 10.

See, U.S. patent application publication Ser. No. 12/146,907(incorporated by reference).

V. Operation and Use

A primary function of AR messages is to convey the information to thegolfer as illustrated in FIGS. 2-5 and also to the other participantsand spectators 48 observing the event. Preferably, a golfer can switchbetween a plan view and the perspective view described herein. A goodexample of such information is distances to a feature, such as a hazardor green or pin placement. In the illustrated embodiment, the AR golfapplication 106 on smart phone 10 is launched and the course information(including feature locations, such as points of interest) and otherinformation is downloaded, if not already stored in memory. In one form,the information can be downloaded remotely over the internet. In anotherform, the information can be downloaded over a cell network. In anotherform, the course information can be downloaded in proximity to thecourse using, e.g., WiFi, Bluetooth, or Near Field Communication (NFC).The course information can be downloaded one time for later re-use, orcan be downloaded over the network of FIG. 6 before or as the round isbeing played.

In any case, the GPS of smart phone 10 provides accurate position of thegolfer 80 when accompanying the golfer (FIG. 2). The distance to the pinor cup on the green 82 can be displayed using the AR message 94 of FIG.3. Of course any arbitrary location on the green can be used toapproximate the position of the cup, such as center of green, front orback of the green, or a grid sector of the green. The GPS location 81 ofthe golfer 80 is compared with the stored location of the target, e.g.pin placement or green center and the difference is computed anddisplayed as at 98. The golfer 80 positions the smart phone 10 so thatthe camera 141 is pointed in the direction of the target, e.g., green 82in FIGS. 2-3. The camera 141 captures the background environment and ARmessages 92, 94 displayed when in the camera field of view. The compassin smart phone 10 senses when the camera 141 is directed toward thepoint of interest and the phone is in an upright orientation (e.g. usingthe gyroscope). The difference computation is displayed as a yardage tothe golfer as AR messages 92, 94, 98 as shown at FIGS. 3-4. For ARmessage 96, the current location 80 is used to look up information froma database in memory and an appropriate pro tip is displayed as at 96.

FIGS. 4 and 5 illustrate different club recommendations based ondifferent golfers. In a preferred example, a golfer loads his personalprofile into device 10, and thereafter a club is recommended after thegolfer pushes a dedicated button 134-136. The profile might specify whatclubs and ball the golfer is using and his shot distribution for eachclub. For example, from range practice the golfer knows he hits his 7Iron between 145 and 165 yards with a 20 yard diameter 50% of the time(155 yard with a 20 yard CEP), i.e., 50% of his 7 iron shots fall withinthe target area 120 illustrated in FIG. 4. Other parameters can be usedas an alternative to CEP (Circular Error Probable or 50%), such as RMS(Root Mean Square, 63-38%) or 2RMS (95-98%), which modify the size(probability) of the circle and shape of the target message 120.Alternatively, different shapes can be used to more accurately reflectthe personal profile if, for example, the along track error is out ofproportion to the cross track error, i.e., the shape of the target 120,124 can be non-circular, such as a rectangle or ellipse. The distance,“155” yards, can be displayed within the target 120 if desired.

Different methods have been devised for recommending how to play a golfhole, but the idea is the object of the game of golf—minimize the scorefor the hole. In a simple example, consider FIG. 4. For the belowaverage golfer of FIG. 4, the method hereof would compute the likelyscore of using a 3 iron from golfer location 81 as 4.78 under Option Abelow, while the likely score using the 7 iron of Option B is 4.34.Therefore, the method hereof recommends option B. This is partlyexplained because the golfer's shot distribution for a 3 Iron fromlocation 81 leads to an end result in the fairway only 50% of the time,while the golfer will end up in the rough 20% of the time and the lake85 30% of the time. (For simplification, this example shows the golfer'spersonal profile as a 60 yard lob wedge and 90 yard sand wedge of aboutequal CEP, putting success close to equal.)

Option A 190 yds- 30 yd CEP Consequence 3 Iron Fairway 50% 1 .5 Hazard30% 2 .6 Rough 20% 1.4 .28 1.38 60 yard Lob Wedge Fairway 70% 1 .7 trap10% 1.2 .12 Rough 20% 1.4 .28 1.1 Total = 2.48 plus 2.3 putts

Option B 155 yds- 20 Yd CEP Consequence 7 Iron layup Fairway 100% 1 1Hazard 0% 2 Rough 0% 1.4 1 Full Sand Wedge Fairway 80% 1 .8 trap 10% 1.2.12 Rough 10% 1.4 .12 1.04 Total = 2.04 plus 2.3 putts

Note that if the golfer of FIG. 4 had the ability (shot distribution) ofthe golfer of FIG. 5 then the method hereof would recommend a 3 Iron.The golfer of FIG. 5 has a 3 Iron profile of 205 yards with a 30 yardCEP. Several different methods for club recommendations have beendeveloped. See, U.S. Publication Nos. 2003/0149496; 2005/0227791;2007/0129178 (incorporated by reference). The selection of algorithm andprobability distribution is a matter of design choice.

Other profiles can be used. For example, the course may have profiledata for “beginner,” “average,” or “good” handicap golfers withoutregard to equipment. Advantageously, whatever profile is used can beadjusted based on factors, such as wind speed and direction, elevationdifference between the target and the golfer location, density altitude,and any equipment differences. Equipment in play might call for slightadjustments based on the type of ball and type of club. Use of network40 of FIG. 6 is particularly useful for recommending clubs based onenvironmental factors. That is, an environmental correction can betransmitted to golfers 48 or club selections can be computed andtransmitted to individual golfers 41.

As illustrated in FIG. 7, advertising or product placements can beinserted as an AR message if desired. For example, FIG. 7 illustrates areplay mode where golfer 80 includes an ad message 86 on his shirt back.Additionally, ad message 88 is inserted on the bag 84. Alternatives arepossible for the placement of the ads, so the message 86 isgeo-referenced to the position of the player 80 using GPS and objectrecognition. That is, the player 80 has a GPS unit 10 on his body andthe ad message 86 is inserted into an AR layer proximate the GPSposition using object recognition for final registration with theplayer's shirt. Bag 84 is identified using object recognition andmessage 88 is placed on bag 84.

FIG. 7 also illustrates a product insert into the AR layer. In FIG. 7,car 92 is inserted into the display in the AR layer. On car object 92,ad message 94 is inserted. AR messages 92, 94 can be displayed near thegreen 82 of FIGS. 3, 4, 5. Such product placement or other AR messagecan occur at convenient geo-referenced locations on the golf course. Admessages 86, 88, and 94 as well as product placement 92, can be insertedinto the perspective views such as FIGS. 3, 4 and 5 where any digitalimage is used as the background.

While a preferred embodiment has been described as using a digitalphotographic image captured by backside camera 141, other digital imagescan be used as a background, such as a virtual rendered background orthe actual image seen through glasses 220 or stored video or panorama.Particularly, for courses where an artificial reality environment isavailable, a virtual background may be preferable as it may facilitatereplay and sharing of event replay within the selected social group.

There are a number of contests that are conducive to use of the methodand system of the present invention. Popular golf games have heretoforebeen limited largely to members of a golfer's playing group. Using thesystem and methods hereof, the “group” can be expanded to otherparticipants 41 and spectators 48, even separated by time and membershipin the social group. For example, select information can be shared orpublished with groups as more fully described in U.S. Pat. No. 7,478,078and U.S. Publication Nos. 2008/0036653; 2011/0142016; 2010/0070758;2011/0098928; and 2009/0144392 (incorporated by reference herein).

A simple “corporate outing” might include contests for members of acorporate group based on: a) closest to the pin on a selected par 3; b)longest drive in the fairway on a selected hole; c) low total; or d) lowhandicap total; etc. Using the system and methods hereof, “outings” arefacilitated. For example “outing” can be composed of ad hoc groupsformed by any golfer that wants to participate with a tee time between10-12. Further, the group might comprise any member of a golfer's socialnetwork group, such as a church group, e.g., that plays onThursday-Saturday in June.

The system and methods hereof allow for enhanced visualization ofcontests, easier logistics, and real time information. For example, agolfer might choose to participate in an “outing” when downloading thecourse information or “checking-in” with the course. Upon arrival at thecontest for “closest to the pin” display 102 will show the golfer on thetee an AR message of the current leader of this contest, as well as theleader's distance from the cup or target.

In another example, a target such as area 120 of FIG. 4 can be displayedto all members participating in an “outing” and the contest is to comeclosest to the center of the target or bulls-eye. Golfers 41participating in the outing “mark” their resulting shot ending location(using a button 134-138) and the results are transmitted through network40 to the golfers in the participating group. Additional AR messages canbe selectively published to golfers participating in the outing and/orspectators 48, such as geo-referenced tags (“Joe hit OB here”) togeneral AR messages (“Joe won target on hole 5 at 8 yards”) toadvertising messages (“Deal of Day for Hole 7—lunch at Maudie'stoday—two people—$12, BUY NOW”). In addition to providing a photo imagebackground view from camera 141, the participant can also provide animage of participant's face using camera 140. This is particularlyuseful if the participant is a pro or celebrity.

In broad form, and in reference to FIG. 12, in establishing an event300, the participant 41 sends an invitation to a social media interestgroup regarding the start of an event as at 302. For example,participant can post the upcoming event to the participant's socialmedia interest group such as participant's Facebook page or by tweetingto a select group, or other methods announcing the about to commenceevent. Before the event, participant 41 selects the application 106 ondevice 10 (FIG. 1) as shown at step 302 of the flowchart 300 of FIG. 12.Members of the social network can elect to become spectators to theevent, preferably in real time.

During the event, spectators can communicate with the participant, aswell as view in perspective the event from the various selectedpositions, such as participant positions. Further, the spectators cansee many, if not all, of the AR messages. Additionally, the spectatorscan award “compensation” to the participant based on the participant'sactivities. Such activities for golf could be hitting a target, asandie, a save, a birdie, etc. Examples of compensation include virtualgoods, virtual services, money, awards, trophies, credits, points, anddonations to a charity.

Example 1: Gallery Application

Bob has a tee time to play a round of golf today. His wife Jane isplaying at the same course earlier in the day and his brother Bill is athome watching the football game.

Both Jane and Bill want to actively follow Bob's golf round.

When Bob gets to the course he activates and logs into the Golf Galleryapplication 106 on his smart phone 10 as seen in FIG. 1. Part of thelog-in process sends a notification through a social media server 44(FIG. 6) to members of Bob's social network to join his “event,” i.e.,creating a virtual gallery for today's round. In this case, Bob'sinvited group is his entire social network.

Another example might have Bob inviting only those in his social networkthat play golf. In still another example, Bob could expand the spectatorgroup to anyone in a defined range of the golf course who has installedthe Golf Gallery application 106.

Jane, who is already on Hole #8 received a notification on her cellphone 10 to join Bob's gallery. She accepts and activates the GolfGallery application 106.

George is a friend of Bob's who has completed his round at the courseand is having lunch in the course restaurant. George is a member ofBob's social network and gets an alert that Bob is at the course andabout to start a round. George chooses to join Bob's gallery byactivating the Golf Gallery application 106.

Bill receives a notification on his iPad while on the couch watching theCowboys/Giants game. Bill chooses to join Bob's gallery by activatingthe Golf Gallery application 106 on his iPad. Additionally, Billresponds with a $10 Challenge BET to Bill that he can't break 90. Thechallenge is communicated through the network of FIG. 6 to Bob's device10.

Bob is able to view who has joined his gallery through the Golf Galleryapplication that is running on his GPS capable smart phone 10. He readsthe Challenge BET notification from Bill and accepts the bet. Thatgenerates a notification to everyone else in the gallery, i.e. thespectators 48 that have elected to observe the event.

Jane get a notification of the Challenge Bet from Bill and Bob'sacceptance. She uses her phone 10 within the application 106 to respondto both with a comment, “You can do it Honey!”

Bob begins his round using his smart phone 10 to capture golf game data(ball position, club used, score as well as environment images). Thecompanion golf application provides him with rangefinder information andtips with AR enhancements as described above.

Jane, George, and Bill are able to view Bob's play, in a realenvironment as captured by camera 41 at each shot. They are also able toview the AR enhancements that are visible to Bob. In alternativeembodiments, the background environment can be artificial which allowsJane, George, and Bill the option to view Bob's play from a number ofperspectives (behind him, from above, from the target/green).

On the first tee Bob records his position and that he is hitting adriver. He them hits his drive 300 yards into the middle of the fairway.When he gets to his ball and records his position by selecting a button(132-136) with the device oriented to the green 82. An AR messagenotifies the gallery (Bill, George and Jane):

-   -   [Bill: Hole #1/Shot 1: 300 yard drive—fairway]

Bob, George and Jane view the notification. Additionally, they can viewthe image just captured by Bob when he recorded his position.

Jane selects an “Applause” option which generates a response to Billthat is displayed in aggregate on his device 10.

Bob selects the “Applause” too and also selects the VIEW option to seeBill's ball position on Hole 1 animated with a virtual or actualimage/environment such as shown in FIG. 3. He sends Bill a message, “YouDa Man . . . Don't hit into that big trap by the green” that is alsovisible to everyone in the gallery.

Bill goes onto to birdie hole #1. When that score is recorded anothernotice is sent to the gallery. Bill also earns a virtual birdie medalthat goes into his library of /trophies. These virtual goods can beconverted into other rewards and incentives. These virtual good andvirtual dollars (ex Facebook credits) can be used to settle bets andpurchase virtual and physical goods and experience.

Jane gets the Birdie Hole #1 notification on her smart phone andshares/posts it with her Facebook network. Five members of her networkread the post and click to join Bob's gallery. They have the option toreview Bill's round history first or pick up his round LIVE. All fivepick the LIVE option.

Bill gets a notification of additional members to his Gallery (now 8total) Bill broadcasts a note to the entire gallery, “I'm gonna breakthe course record today”

On Hole 3, Bill is having trouble deciding which club to hit. Hebroadcasts a POLL to the entire gallery:

-   -   [150 yards to pin on Hole 3, all carry, 5 mpg wind at my back:        which club to hit: 1.) 8 iron 2.) 9 iron]

The poll broadcast notification also includes a promotional offer fromthe course for a discounted round and a link to their website.

Three members of the gallery respond, two choose 8 iron and one chooses9 iron. Bob get the notification and hits his 8 iron onto the green.

Bill is watching Bob's round with concern because it looks like Bob ison track to shoot a low score, which will cost him the bet. Bill knowsthat Bob prides himself on hitting long drive which can get him intotrouble. Bill initiates a COURSE CHALLENGE event on hole #9 which is along par 5.

Bill issues/inputs a Challenge that if Bob can hit a 325+ yard drive inthe fairway he wins $10.

When Bob gets to Hole #10 the Challenge is presented to him and heaccepts. On Bob's view of the hole the AR functionality now shows himthe target area he needs to hit into to WIN the challenge. See, forexample, targets 120 and 124 in FIGS. 4-5.

Bob over swings and pulls his drive left into the rough. When he get tohis ball and records his position, he and the entire gallery get thenotification that he has lost the challenge.

Bill responds with a snarky response: “Sucker”

Jane responds to Bill: “Leave my husband alone”

Bob comes to the 12^(th) hole which today is part of a promotional longdrive contest that club manufacturer Taylor Made is sponsoring at over500 participating courses nationwide. The current long drive of the dayis displayed for Bob using AR functionality, such as a target 120, 124in FIGS. 4-5. Bob smacks a 328 yard drive, when he marks his ball hisdistance is registered and a notice is created announcing that Bob isnow the current long drive for both the course and the state of Texas.Another virtual medal is awarded to his trophy case. Both are broadcastto his gallery along with a link to the contest leaderboard.

FIG. 12 is a flow chart illustrating an overview of the “gallery”example described above. In establishing an event 300, the golfer Bobsends an invitation to his social media interest group regarding thestart of his round as at 302. In this example, the social media interestgroup is Bob's Facebook page by tweeting or posting the about tocommence golf round. Before he tees off, Bob selects the application 106on device 10 (FIG. 1) as shown at step 302 of the flowchart 300.

In this example, Bill, Jane and George all elect to “subscribe” to theevent posted by Bob, as shown at 306. During the golf round, Bill, Jane,George and Bob all interact as described above and shown at block 308 ofFIG. 12. While the golf round is viewed “live” in real time in thisexample, the round can be replayed by any spectator or participant asshown at 310.

Example 2: Sports Event Participation & Management Application

The Seven Rivers Golf Club in Arlen, Tex. is one of 50 courses hosting acharity golf tournament as part of a national event to raise money forCancer Research.

Forty golfers will participate in ten “foursome” on Saturday at SevenRivers. The other courses will host similar single day events onsuccessive weekends.

As each golfer arrives at Seven Rivers for the tournament check-in, theyhave the option of launching the application 106 on their GPS enabledsmart phone 10, or opt to use a cart-mounted GPS enabled iPad withwireless functionality in their cart.

Every golfer 41 in proximity of the course that has application 106installed on their phone 10 will get a message: “Are you playing in thetournament today?” which will launch the “check-in” to join the event byactivating application 106.

The application 106 check-in process identifies the individual using theregistration information entered into application 106. The application106 queries the social server 44 and identifies the team he is playingon along with the course and the event, which in this case is linked tothe larger event that covers 50 courses and up 2000 golfers and 500teams.

As detailed in other areas of the application, each golfer 41 will usethe application 106 to record his and his teams performance, experienceAR enabled information and game enhancements, social enhancementsthrough the “gallery” functionality.

The activity data from every participant 41 in the tournament isrecorded, consolidated and stored by a central server. This supportsreal time access by all participants 41 to LEADERBOARD functionality andthe ability to dial down to current data on any course, team orindividual.

The previous described Gallery functionality is now augmented by theability to follow a particular team. This allows the course andtournament the ability to monitor and communicate (individually andcollectively) to participants 41 wherever they are on the course.

Similarly the AR messages illustrated in FIGS. 3-5 can now be targetedto an individual, team(s), a location/course or an entire event.

Example 3: Spectator Following Celebrity/Pro Golfer

The National Charity Golf Event for Cancer Research has a number ofprofessional golfers participating including 2011 US Open Champion RoryMcIlroy.

McIlroy has announced to his social network on all the platforms he uses(Twitter, Facebook, Google+, YouTube) that he will be providing unique“inside the ropes virtual access” of his round in exchange for adonation to the Cancer Research organization. The Cancer Researchorganization also communicates this opportunity as part of theirmarketing campaign to promote the event.

Over 50,000 virtual spectators sign up to participate. Half pay the $5fee with Paypal, the other half pay using a virtual social currency likeFacebook credits. Each receives a digital badge keepsake for theirdonation.

McIlroy's caddie is outfitted with a head mounted camera and microphonethat enables real-time streaming of image and voice content to thesocial media server. The video stream includes AR messages as describedherein. The camera and audio control resides with McIlroy but someplayers allow the spectators in the gallery to control when the imageand audio stream is on or off.

Similar to the previously described gallery example 1, spectators areable to view/participate in McIlroy's round with AR messages, seeperspective views from McIlroy's current position and preferably selecta perspective view from a number of different viewing locations orviewpoints.

On the 2nd hole, McIlroy has his caddie upload the following offer, “$5donation if I can hit this drive to within 10 feet of the cup” that isbroadcast to the entire gallery through the social media server. 5,000members of the virtual gallery opt-in to the challenge.

McIlroy hits his 7 iron 2 feet from the cup. 40,000 members of thevirtual gallery hit the applause button. 5,000 members are charged $5yielding an additional $25,000 for the Cancer Research charity. Uponprocessing of their payment they each receive a digital badge “I betagainst Rory and lost for a great cause”.

Example 4: Ski Event

Turning to FIG. 10, the system and methods hereof are described using aski race as an example of an event. Before the race, skier 400 invitesspectators, such as a group or circle interested in skiing, in hersocial network to join or participate in her traversal of ski course 402(sometimes referred to herein as “event venue”). In this embodiment,skier is wearing the device 10 (not shown), the goggles 220 of FIG. 8,and a helmet camera 404. The camera 404 is preferably a wireless cameraconnected by Bluetooth to device 10. Examples of such cameras areJonesCAM LX, Vuzix iWear CamAR available from Vuzix Corporation,Rochester, N.Y., AT-1Wireless available from Dogcam, and ContourGPSavailable from Contour HD.

In the embodiment of FIG. 10, the skier 400 invites spectators and thenselects application 106 on device 10. The skier 400 then activatescamera 404 to stream video to device 10 worn by the skier 400. The videois then streamed from device 10 to social media server 44 as shown inFIG. 6. In addition to the video, the device 10 transmits periodic GPSposition data to the social media server 44. In this example, thebackground environment is preferably an artificial reality 3D model ofthe ski course 402 as described above. The skier 400 is represented byavatar traversing the ski course on the model. Thus, spectators viewingthe ski race can track the progress of the skier 400 down course 402from any number of spectator selected viewpoints, including theviewpoint of the skier 400, as well as select video feeds from helmetcamera 404.

In addition, skier 400 has access to augmented reality overlays of thecourse 402 using goggles 220. In this example, spectators are notcommunicating with the skier during the run. However, a blue “ideal”fall line or traversal of the gates as well as a pace indicator are ARmessages seen on the goggles 220 by skier 400.

Example 5: Bicycle Event

Turning to FIG. 11, a number of bicycle riders is shown, with rider 410being the participant in this example. Of course, all riders could beparticipating with the same or different social networks or subsets orinterest groups therein. In this embodiment, rider 410 is wearing thedevice 10 in a jersey pocket (not shown), the goggles 220 of FIG. 8, anda helmet camera 404.

In the embodiment of FIG. 11, the biker 410 invites spectators and thenselects application 106 on device 10. The biker 410 then activatescamera 404 to stream video to device 10 worn by the biker 410. The videois then streamed from device 10 to social media server 44 as shown inFIG. 6. In addition to the video, the device 10 transmits periodic GPSposition data to the social media server 44. In this example, thebackground environment is preferably an artificial reality 3D model ofthe bike course as described above. The biker 410 is represented byavatar traversing the bike course. Thus, spectators viewing the bikecourse can track the progress of the biker 410 along the course from anynumber of viewpoints, including the viewpoint of the biker 410, as wellas select video feeds from helmet camera 404.

In addition, biker 410 has access to augmented reality overlays of thecourse using goggles 220. In this example, selected spectators and thebiker's coach can communicate with biker 410 during the race event. Forexample, the coach can dictate strategy and indicate other participantsto pass or draft from, etc. Further, the AR messages can include grade,preferred path information, elapsed time, distance traveled, distance togo, actual time, actual pace, desired pace and any other desiredperformance parameters or bike sensor data. Some or all AR messages canbe shared with some or all of the spectators.

Example 6: Other Events

While described above for a golf event, ski event, and bike event, thesystems and methods hereof can also be used for a variety of events,such as walk/running or hiking events, biking events, BMX, surfing,skate board, boating/sailing events, fishing, hunting, driving events,kayaking, ski diving, any powered events including aerobatics, etc.Thus, while an event is preferably a participatory sporting event, itcan include recreational or social events such as hiking or sightseeing.

What is claimed:
 1. A method of viewing a golf event in real time duringplay, comprising: determining the position of a golfer on a virtual golfvenue during play; comparing the position of a virtual feature on saidvirtual golf venue with said golfer position; operating glasses, havinga display, worn by the golfer to view said virtual feature in aperspective view from said golfer position; viewing, on the glassesdisplay, in real time during play an artificial reality (“AR”) messagegeo-referenced to a virtual location or virtual object at said virtualgolf venue in said perspective view on said virtual golf venue, whereinsaid perspective view is not an overhead view of said golf event.
 2. Themethod of claim 1, wherein said AR message comprises a distance overlaidat least a portion of the virtual golf venue as a backgroundenvironment.
 3. The method of claim 1, wherein said AR message isgeo-referenced to at least a portion of a virtual environment of thevirtual golf venue as a background environment.
 4. The method of claim1, wherein said AR message is transmitted to spectators.
 5. The methodof claim 1, wherein changing the view of said AR message compriseschanging the view point to the location of a feature.
 6. The method ofclaim 1, including additional participants wearing glasses, saidadditional participants viewing a perspective view of the golf venue. 7.The method of claim 1, including changing the view to an overhead planview.
 8. The method of claim 1, wherein a spectator performs saidviewing the AR message step.
 9. Glasses wearable by a golfer during theplay of golf comprising: a memory storing the position of the golfer ata virtual golf venue; a memory storing the location of one or morevirtual features of said virtual golf venue; a processor operable forcomparing said golfer position with the location of a virtual feature;and a display operable to view a perspective view of said virtual golfvenue during play from said golfer position at said virtual golf venueto said virtual feature, said display operable to show an artificialreality (“AR”) message on a virtual background, and wherein said displayis operable to view at least a portion of said virtual golf venue aspart of said virtual background and to view said artificial realitymessage geo-referenced to a virtual location or virtual object of saidportion.
 10. The glasses of claim 9, wherein said augmented realitymessage includes an indicator of the location of the feature in saidgolfer's perspective view of said golf venue.
 11. The glasses of claim9, wherein said feature is a location on a green at the golf venue beingplayed.
 12. The glasses of claim 9, wherein said artificial realitymessage is geo-referenced to an object.
 13. The glasses of claim 9,wherein said perspective view background of said golf venue comprising avirtual environment.
 14. The glasses of claim 9, wherein perspectiveview background of said golf venue includes an artificial rendering. 15.The glasses of claim 9, including a tethered computer wherein a memoryincludes a map of the golf venue being played, and said golfer canoperate the glasses to display a plan view of said map on said display.16. The glasses of claim 9, wherein a memory includes a database havingfeature locations and a plurality of advertisements.
 17. The glasses ofclaim 16, wherein one of the plurality of advertisements is downloadedfrom the database and viewed on said display to overlay said perspectiveview.
 18. The glasses of claim 17, wherein one of the plurality ofadvertisements is downloaded based in part on information related tosaid golfer.
 19. A method of using glasses worn by a golfer during playof a virtual golf venue, comprising: determining a position of saidgolfer on said virtual golf venue; comparing said golfer position with avirtual target location of said virtual golf venue to determine adistance between said golfer position and target location; displaying onthe glasses, a perspective view of said virtual golf venue from saidgolfer position on said virtual golf venue towards said virtual targetlocation, wherein said perspective view is not an overhead plan view;and displaying an artificial reality (“AR”) message geo-referenced to avirtual location or virtual object on said perspective view includingsaid distance to approximately the virtual target location.
 20. Themethod of claim 19, including retrieving from a memory a virtualenvironment of said golf venue and displaying the virtual environment asat least part of the background on the glasses.
 21. The method of claim19, including inserting an advertising message on said perspective view.22. The method of claim 19, wherein said virtual golf venue is a rangehaving a number of targets.
 23. The method of claim 19, said glassesincluding a tethered device, said glasses being connectable to a server.24. The method of claim 19, said AR message comprising text from amember of the golfer's social network.
 25. The method of claim 19,including a virtual product placement on said perspective view.
 26. Amethod of displaying messages on glasses worn by a golfer during play ofgolf at a virtual golf venue, comprising the steps of: connecting theglasses with a server establishing communication between said glassesand said server; receiving information concerning a feature of said golfvenue on said glasses from said server; viewing on the glasses duringplay of said virtual golf venue a perspective view from a position ofthe glasses on said virtual golf venue towards said feature on saidvirtual golf venue, including viewing said feature on a display of theglasses; and viewing a geo-referenced, artificial reality (“AR”) messagegeo-referenced to a virtual location or virtual object on saidperspective view showing said information of said feature on saidvirtual golf venue.
 27. The method of claim 26, wherein said serverincludes demographics and said AR message includes an advertisementdeemed relevant to such demographics.
 28. The method of claim 26,wherein said server includes information personal to said golfer andsaid AR message includes an advertisement deemed relevant to saidpersonal information.
 29. The method of claim 26, wherein said ARmessage includes an icon and the golfer can activate a purchase byselecting said icon.
 30. The method of claim 26, wherein said AR messageincludes a coupon that can be selected and stored for later use on saiddevice.